Vehicle auxiliary display device, display method and vehicle

ABSTRACT

A vehicle auxiliary display device includes: a first camera configured to acquire a first image of a position of a user; a second camera configured to acquire a second image within a first range on an outside of a pillar of a vehicle; and a processor configured to determine a shielding area of the pillar shielding the user&#39;s line of sight based on the first image, extract an image of the shielding area from the second image based on the second image, and send the image of the shielding area to the user.

TECHNICAL FIELD

Embodiments of the present disclosure relate to a vehicle auxiliarydisplay device, a display method of the vehicle auxiliary displaydevice, and a vehicle comprising the vehicle auxiliary display device.

BACKGROUND

At present, cars, as one kind of the most commonly used transport means,have entered millions of households. With the popularization of cars,the driving safety problem is also getting more and more attention. Oneof the most common safety problems involves the blind areas formed by Apillars of a car. The A pillars are connecting pillars on the front leftside and the front right side of the car for connecting the roof and thefront cabin. For a car, pillars such as the A pillars not only have asupporting function but also play the role of door frame. However, thepillars produce visual blind areas for drivers, and this effect due tothe A pillars is the most obvious. There is a dilemma for the width ofan A pillar (affecting the cross section thereof). If the width is toosmall, the bearing capacity in unexpected situations will beadvantageously affected, but if the width is too large, the user's fieldof view will be advantageously affected, which situations may thereforebe detrimental to safety.

SUMMARY

The objectives of the embodiment of the present disclosure is to providea vehicle auxiliary display device, a display method thereof and avehicle comprising the same.

At least an embodiment of the present disclosure provides a vehicleauxiliary display device, comprising: a first camera configured toacquire a first image of a position of a user; a second cameraconfigured to acquire a second image within a first range on an outsideof a pillar of a vehicle; and a processor configured to determine ashielding area of the pillar shielding the user's line of sight based onthe first image, extract an image of the shielding area from the secondimage based on the second image, and send the image of the shieldingarea to the user.

For example, the first camera is configured to acquire the first imageaccording to a first space determined by an area provided with a maindriving seat and a height of an A pillar as the pillar in a directionperpendicular to a plane provided with the vehicle.

For example, the second camera is configured to acquire the second imageaccording to a second space defined by first extension lines ofconnecting lines of at least part of vertexes of the first space and atleast part of vertexes of the A pillar.

For example, the first image acquired by the first camera includes animage of the user's eyes; and the processor determines a position of theuser's eyes based on the first image, determines a coordinate of theuser's eyes in the first space based on the position of the user's eye,and determines the shielding area of the A pillar shielding the user'sline of sight based on the coordinate.

For example, the processor determines second extension lines ofconnecting lines from the coordinate to the vertexes of the A pillar,and determines the shielding area of the A pillar shielding the user'sline of sight based on the second extension lines.

For example, the processor determines vertexes of the shielding area inthe second image according to the second extension lines, and extractsthe image of the shielding area from the second image based on thevertexes of the shielding area.

For example, the second camera is configured to acquire the second imageby forming a first angle with respect to the direction perpendicular tothe plane provided with the vehicle.

For example, the device further comprises a display configured todisplay the image of the shielding area.

For example, the display is configured to form a second angle withrespect to the direction perpendicular to the plane provided with thevehicle; the display is disposed on the A pillar, so that the image ofthe shielding area displayed on the display can be inclined relative toan edge of the display but not inclined relative to the directionperpendicular to the plane provided with the vehicle.

For example, the first angle, the second angle and an angle of the Apillar relative to the direction perpendicular to the plane providedwith the vehicle are all equal.

At least an embodiment of the disclosure further provides a vehicle, thevehicle comprises pillars and the vehicle auxiliary display device, andone of the pillars is equipped with a display of the vehicle auxiliarydisplay device.

For example, the pillars include A pillars; and the vehicle auxiliarydisplay device is disposed on at least one A pillar on the front of thevehicle.

For example, the at least one A pillar includes a body; the bodyincludes a barrier part and a safety airbag; and the barrier part isconfigured to isolate the safety airbag from the display.

For example, the at least one A pillar further includes a shield; andthe shield is capable of covering the body of the A pillar and is ableto be turned on or off relative to the body of the A pillar.

For example, an opening is disposed at a position of the shieldcorresponding to the display and configured to expose a display area ofthe display.

At least an embodiment of the disclosure still further provides adisplay method of a vehicle auxiliary display device, the vehicleauxiliary display device comprising a first camera and a second camera,the display method comprising: adopting the first camera to acquire afirst image of a position of a user; adopting the second camera toacquire a second image within a first range on an outside of a pillar;determining a shielding area of the pillar shielding the user's line ofsight based on the first image, and extracting an image of the shieldingarea from the second image based on the second image; and sending theextracted image of the shielding area to the user.

For example, the first camera is adopted to acquire the first imageaccording to a first space determined by an area provided with a maindriving seat and a height of an A pillar as the pillar in a directionperpendicular to the plane provided with the vehicle.

For example, the second camera is adopted to acquire the second imageaccording to a second space formed by first extension lines ofconnecting lines of at least part of vertexes of the first space and atleast part of vertexes of the A pillar.

For example, the first camera is adopted to acquire the first imageincluding an image of the user's eyes, determine the position of theuser's eyes based on the first image, determine a coordinate of theposition of the user's eyes in the first space based on the position ofthe user's eye, and determine the shielding area of the A pillarshielding the user's line of sight based on the coordinate.

For example, second extension lines of connecting lines from thecoordinate to the vertexes of the A pillar are determined, and theshielding area of the A pillar shielding the user's line of sight isdetermined based on the second extension lines.

For example, vertexes of the shielding area in the second image aredetermined according to the second extension lines; and the image of theshielding area is extracted from the second image based on the vertexesof the shielding area.

For example, the second camera is configured to acquire the second imagerelative to the direction perpendicular to the plane provided with thevehicle.

For example, the display method further comprises: displaying the imageof the shielding area on a display on the pillar.

For example, the display is configured to form a second angle withrespect to the direction perpendicular to the plane provided with thevehicle, so that the image of the shielding area displayed on thedisplay can be inclined relative to an edge of the display but notinclined relative to the direction perpendicular to the plane providedwith the vehicle.

For example, the first angle, the second angle and an angle of the Apillar relative to the direction perpendicular to the plane providedwith the vehicle are all equal.

In the embodiment of the present disclosure, external conditioncorresponding to the blind area of the A pillar is reflected on adisplay of the A pillar in real time after image processing according tothe observing position of the human eyes, so the perspective effect ofthe A pillar can be achieved, and hence the driving safety can beimproved.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to clearly illustrate the technical solution of the embodimentsof the disclosure, the drawings of the embodiments will be brieflydescribed in the following; it is obvious that the described drawingsare only related to some embodiments of the disclosure and thus are notlimitative of the disclosure.

FIG. 1 is a schematic structural view of car pillars in the embodimentof the present disclosure;

FIG. 2 is a schematic structural view of a vehicle auxiliary displaydevice provided by an embodiment of the present disclosure;

FIG. 3a is a side view of a car in an embodiment of the presentdisclosure;

FIG. 3b is a top view of the car in the embodiment of the presentdisclosure;

FIG. 3c is a schematic diagram illustrating the positional relationshipof a first space;

FIG. 4 is a schematic diagram illustrating the external shape of the Apillar in an embodiment of the present disclosure;

FIGS. 5a-5g are schematic diagrams illustrating the monitoring modes ofa second camera in an embodiment of the present disclosure;

FIG. 6 is a schematic structural view illustrating the inside of the Apillar of the car in an embodiment of the present disclosure; and

FIG. 7 is a flow diagram of a display method of a vehicle auxiliarydisplay device provided by an embodiment of the present disclosure.

DETAILED DESCRIPTION

Detailed description will be given below to the preferred embodiments ofthe present disclosure with reference to the accompanying drawings. Itshould be noted that: in the description and the accompanying drawings,basically same steps and elements are marked by same reference numeralsof the accompanying drawings, and the repeated description of the stepsand the elements is omitted.

In the following embodiments of the present disclosure, “vehicle” refersto a non-rail-supported vehicle having three or more wheels, beingdriven by power and for carrying people and/or freight, for instance,including non-convertible motor tricycles, cars, trucks, etc.Description is given in the embodiment of the present disclosure bytaking the car as an example.

FIG. 1 is a schematic structural view of car pillars in an embodiment ofthe present disclosure. As shown in FIG. 1, a car 100 includes threekinds of pillars which are sequentially an A pillar 110, a B pillar 120,and a C pillar 130 from front to back. Although only pillars on the leftside are shown in FIG. 1, it should be understood by those skilled inthe art that the A pillar, the B pillar and the C pillar are alsocorrespondingly disposed on the right side of the car. The vehicleauxiliary display device and the display method, provided by theembodiment of the present disclosure, not only can be applicable to an Apillar to solve the blind area problem of the A pillar but also can beapplicable to a B pillar and a C pillar to solve the blind area problemof the B pillar and the C pillar, for instance, displaying shieldingareas of the B pillar and the C pillar while reversing or parking.Description will be given below to the system and the method provided bythe embodiment of the present disclosure by taking the A pillar only asan example. The technical proposals for solving the blind area problemof the B pillar and the C pillar can be also realized by those skilledin the art according to similar methods.

FIG. 2 is a schematic structural view of a vehicle auxiliary displaydevice provided by an embodiment of the present disclosure. Asillustrated in FIG. 2, the vehicle auxiliary display device 200comprises a first camera 210, a second camera 220 and a processor 230.The processor 230 can be a general purpose processor, e.g., a centralprocessing unit (CPU), and may also be a special purpose processor,e.g., a programmable logic circuit (PLC).

According to one example of the present disclosure, the vehicleauxiliary display device may further comprise a display 240. The display240 is disposed on one pillar (e.g., an A pillar) of a car. A displayunit 320, for instance, may be a liquid crystal display (LCD) or anorganic light-emitting diode (OLED) display and may also have touchfunction. Of course, alternatively, the display may also be not used,but virtual reality (VR) or augmented reality (AR) glasses or projectoris used.

According to one example of the present disclosure, the first camera 210is configured to monitor and acquire a first image of the position of auser. The acquisition methods, for instance, may involve that theacquisition is conducted facing the position of the user and may also beconducted in an oblique direction, not facing the position of the user.The user, for instance, is the driver for driving the vehicle. Thesecond camera 220 is configured to acquire a second image within a firstrange outside the car pillar. The first range at least includes an areashielded by the pillar. The processor 230 is configured to determine ashielding area corresponding to the car pillar shielding the user's lineof sight based on the first image, extract an image of the shieldingarea from the second image based on the second image, and send the imageto the user for viewing. For instance, the image is sent to the display240 for display. Correspondingly, the display 240 is configured todisplay the image of the shielding area of the pillar. Moreover, forinstance, for the user to view the area shielded by the pillar, theimage of the shielding area can further be projected onto the pillar viaa projector. Of course, when the user wears VR or AR glasses, the imageof the shielding area may also be sent onto the VR or AR glasses.

In the example of one embodiment of the present disclosure, the carpillar provided with the display is the A pillar. In order to solve thetechnical problem of the blind area of the A pillar, for instance, thefirst camera 210 may monitor and acquire the first image according to afirst space determined by the position of the main driver's seat and thevertical height of the A pillar. For instance, the second camera mayacquire the second image by determining the monitored first rangeaccording to the relative positional relationship of the first space andthe A pillar. For instance, when the first space is set to be arectangle, a second space (space range) is determined as the monitoredfirst range according to the vertexes of the rectangle and the vertexesof the A pillar.

FIGS. 3a-3b are schematic diagrams illustrating monitoring areas of thefirst camera and the second camera in the example of the embodiment ofthe present disclosure. FIG. 3c is a schematic diagram of the firstspace. FIG. 4 is a schematic diagram illustrating the external shape ofthe A pillar in the embodiment of the present disclosure. Descriptionwill be given below to the position of the monitoring areas of the firstcamera and the second camera with reference to FIGS. 3a-3c and 4.

In order to accurately determine the visual blind areas of the user(namely areas that is outside of the car and cannot be viewed by theuser in the car due to the shielding of the pillar), the position of theuser in the car is determined at first. As shown in FIG. 3a , taking anA pillar on the left of a left rudder car as an example, the height ofthe A pillar of the car may be the vertical distance from the top of theA pillar (the position intersected with the ceiling of the car) to thebottom (for instance, the position intersected with the lower edge ofthe window), and this vertical distance is the length of a vertical linec in FIG. 3a . In addition, straight lines X and Y in FIG. 3aillustratively provide the front view range e when the user looksstraight ahead.

In addition, the user sits in the main driving position while driving,so the possible position of the user in the car can be determined by,for example, obtaining an area where the main driving seat is located.As shown in FIG. 3b , an upper surface of a main user seat is generallyregarded as a rectangle or a rectangle-like shape, e.g., a rectangle ora square. The corresponding four vertexes of the rectangle on a bottomhorizontal plane in the height direction of the A pillar (namely thehorizontal plane where the bottom of the line c ends) are respectivelymarked by characters: h, i, j and k, and the corresponding four vertexesof the rectangle on a top horizontal plane in the height direction ofthe A pillar (namely the horizontal plane where the top of the line cends) are respectively marked by characters: H, I, J and K as shown inthe drawing. Thus, the eight vertexes form a rectangular space as shownin FIG. 3c . The height of the rectangular space is as described aboveand is the height c of the A pillar. The length of the rectangle alongthe length direction of the car is the length a of the rectangular seat,and the width along the width direction of the car is the width b of therectangular seat. That is to say, the distance between H and I or thedistance between J and K is a, and the distance between H and J or thedistance between I and K is b. Because the body of the user generallydoes not move from the position of the main user seat while driving, theuser's eyes are within the range of the main driving seat determined bythe rectangle. Moreover, as the user needs to view the traffic aroundthrough a front windshield as well as the left and right windshields(windows), the position of the user's eyes is within the space range.Therefore, the space range of the user's eyes is a rectangular areaformed in the height direction of the A pillar, and the length, thewidth and the height are respectively a, b and c. The rectangular areais defined as the first space, namely the area to be monitored andcaptured by the first camera.

As shown in FIG. 3c , the first space includes eight vertexes H, I, J,K, h, i, j and k. H, I, J and K are four vertexes on the top in theheight direction of the A pillar, and h, i, j and k are four vertexes atthe bottom in the height direction of the A pillar. The first camera cancapture the position of the first space when mounted on the car, forinstance, may be mounted on the inside or the outside of the car, morespecifically, for instance, may be mounted at an area above or aside afront windshield on the inside of the car or above a windshield on theoutside of the car, so that the first camera can randomly acquire theposition of the user, particularly the first image of the position ofthe user's eyes.

Beside that the first camera is adopted to acquire the position of theuser, the second camera is adopted to capture the second image withinthe first range on the outside of the corresponding A pillar of the car.The first range at least includes a visual blind area of the user causedby the A pillar. For instance, the second camera may be placed at aposition where the second image within the first range on the outside ofthe car can be acquired, for instance, on the inside or the outside ofthe car, e.g., at a position of a rear-view mirror of the car or on aside surface of the front windshield.

As the body of the user generally does not move from the first space inthe driving process, for the purpose of convenient determination of anexternal area of the car where the user's line of sight is shielded bythe A pillar, in the example of the embodiment of the presentdisclosure, a predetermined space range on the outside of the car pillarof the second image acquired by the second camera, namely a secondspace, can be determined according to the border of the first space, forinstance, the vertexes of the first space and the vertexes of the Apillar. Description will be given below to one example of how todetermine the second space based on this method.

As described above, the length, the width and the height of the firstspace are respectively a, b an c; the first space is a rectangular areaformed in the height direction of the A pillar and includes eightvertexes; part of the vertexes or all the vertexes therein can beselected to lead straight lines towards part of the vertexes or all thevertexes of the A pillar, so as to determine the second space includingthe shielding area on the outside of the car where the user's line ofsight is shielded by the A pillar. For instance, the right front vertexand the left rear vertex among the 8 vertexes may be selected as twovertexes of a diagonal, so as to determine the maximum range of thesecond space as much as possible. Moreover, for instance, 4 vertexes onthe upper part of the first space may be selected to lead straight linestowards 2 vertexes on the upper part of the A pillar, or 4 vertexes onthe lower part of the first space may also be selected to lead straightlines towards 2 vertexes on the lower part of the A pillar. Of course,the 8 vertexes may also be adopted to respectively lead straight linestowards the 4 vertexes of the A pillar, and the most marginal extensionline is selected to form the second space.

FIG. 4 is a schematic diagram illustrating the shape of the left front Apillar of the car (the rear-view mirror on the left is included in thefigure). The surface of the A pillar on the left observed by the maindriver sitting at the main driving position inside the car is as shownin FIG. 4. The surface is, for instance, an irregular curved surface.The curved surface includes four vertexes O, P, Q and R, in which P andR points are in the front, and O and Q points are in the rear. The shapeof the A pillar on the right may be the same as that of the A pillar onthe left, although the positions relative to the driver are different.

With reference to FIGS. 3b and 4, in the example, two vertexes among theeight vertexes of the first space may be selected to lead straight linestowards two vertexes of the A pillar. For instance, an upper vertex onthe left rear side of the first space, namely the K point, is selectedat first, and then an upper vertex on the right front side of the firstspace, namely the H point, is selected. The K point is connected with anupper vertex on the right front side of the A pillar, namely the Ppoint, to obtain a horizontal straight line M as shown in FIG. 3b ; andthe H point is connected with an upper vertex on the left front side ofthe A pillar, namely the O point, to obtain a horizontal straight line Nas shown in FIG. 3b . The straight lines M and N are in the samehorizontal plane. The area (range) between the lines M and N can betaken as a horizontal range d monitored by the second camera and theheight range may be selected as the car height.

Further alternatively, with reference to FIGS. 3b and 4, an upper vertexon the left front side of the first space, namely the J point, which isthe closest to the A pillar on the left, is selected. The J point isconnected with the upper vertex on the right front side of the A pillar,namely the P point, to obtain a horizontal straight line. The J point isconnected with the upper vertex on the left front side of the A pillar,namely the O point, to obtain another horizontal straight line. The twostraight lines are also in the same horizontal plane. The area (range)between the two straight lines can be taken as a horizontal range dmonitored by the second camera, and the height range may be selected asthe car height.

The space determined according to the horizontal range and the heightrange may be defined as the second space which covers the shielding areaon the outside of the car where the line of sight is shielded by the Apillar, namely the visual blind area. It should be understood by thoseskilled in the art that the method for determining the range of thesecond space captured by the second camera is not limited to the aboveexample, as long as the second space can cover the shielding area ofcorresponding car pillar for shielding the user's line of sight.

After determining the second space, the second camera may be disposed atone position, where the second image of the second space may beacquired, so that the acquired second image can include the visual blindarea without excluding any line-of-sight area which is shielded.

Because the area captured by the second camera for the second image ofthe second space is larger than the visual blind area, if the entiresecond image is directly displayed to the user, it is helpful for userobservation as a whole. In order to obtain the image of the shieldingarea of the A pillar shielding the user's line of sight from the secondimage, the processor 230 may determine the shielding area of the Apillar shielding the user's line of sight based on the first image, andsubsequently further extract the image of the shielding area from thesecond image based on the second image.

In order to determine the shielding area of the A pillar of the car thatshields the user's line of sight, the position of the user's eyes, theposition of a center point (the space between the eyebrows) of aconnecting line of two eyes, or the like may be determined at first. Inan embodiment of the present disclosure, the position of the user's eyescan be determined according to the first image acquired by the firstcamera. For instance, the image of the user's eyes is extracted by wayof image recognition, and subsequently a coordinate of the eye positionin the first space is determined. A coordinate system in the firstspace, for instance, may adopt a vertex in the first space that is theclosest to the bottom of the A pillar as an origin (namely the j point),adopt the direction parallel to the car width as an X axis, adopt thedirection parallel to the car length as a Y axis, and adopt the heightdirection as a Z axis. The coordinate values of the user's eyes in thedirections of the X axis, the Y axis and the Z axis are determined, andthen the coordinate of the user's eyes in the first space aredetermined. In addition, for the convenience of obtaining the positionsof the user's eyes by the first camera, for instance, part of referencepoints may be provided inside the car body for helping imagerecognition.

After the coordinate of the user's eyes are determined, the shieldingarea of the A pillar may be determined according to the coordinates andthe dimension parameters of the car. The dimension parameters of the carinclude the size of the car seat, the shape and the size of the A pillarof the car, the relative position between the car seat and the A pillar,etc. The dimension parameters may be fixed or measurable for each carmodel and hence can be easily obtained.

With reference to FIG. 3c , supposing that the user's eyes are disposedat a Z point and the coordinate of the Z point is (x1, y1, z1), theshielding area of the A pillar of the car that shields the user's lineof sight is determined based on the coordinate of the Z point. Accordingto one example of the present disclosure, the processor 230 determinesextension lines from the coordinate Z to the A pillar, for instance,extension lines to the vertexes of the A pillar, so as to define a spacerange similar to a pyramid. For instance, the space range on the outsideof the car is defined according to extension lines of straight linesfrom the coordinate Z to the vertexes O, P, Q and R of the A pillar. Thespace range is regarded as the shielding area of the user relative tothe A pillar.

Subsequently, the processor 230 extracts the image of the shielding areafrom the second image based on the second image. For instance, the imageof the shielding area is segmented from the second image that is formedthrough the acquisition of the second space by the second camera 120.For instance, the coordinates of the extension lines of the straightlines from the Z point of the user's eyes in the first space to thevertexes O, P, Q and R of the A pillar, mapped into the second image,may be determined at first, and the image of the shielding area isextracted from the second image according to the coordinates. Forinstance, after the coordinate Z is determined, the distances ofconnecting lines from the Z point in the first space to various vertexesH, I, J and K of the first space, the distances of connecting lines fromthe Z point to the vertexes of the A pillar, and the angles among theconnecting lines can be determined. Subsequently, coordinate values ofthe coordinates of the extension lines from the Z point to the vertexesof the A pillar can be determined based on geometric calculationaccording to the position and the shooting range of the second cameraand the predetermined directions of the extension lines of the secondspace and mapped into the second image, and the image of the shieldingarea is extracted from the second image according to the coordinatesobtained through mapping.

The processor 230 may send the image of the shielding area onto thedisplay 240 for display after the segmentation treatment of the image ofthe shielding area. Thus, the user can know the condition of theshielding area behind the A pillar, which area is originally shielded bythe A pillar, according to the image displayed on the display 240.

For instance, the process of extracting the image of the shielding areaby the processor 230 can also be implemented in real time. When the eyeposition changes due to the change of the sitting posture of the user,the image of the shielding area segmented by the processor 230 can alsochange correspondingly according to the change of the viewing angle ofthe user. Thus, the user can more conveniently observe the condition inthe shielding area behind the A pillar, which area is originallyshielded by the A pillar.

In the embodiments of the present disclosure, the method of acquiringthe first image, the second image and the image of the shielding area isnot limited to the above ways. The present disclosure is not limited tothe specific ways. For instance, the center point of the upper surfaceof the first space (namely a plane defined by the vertexes JHIK) isselected; the center point of the A pillar (for instance, the crosspoint of the connecting line connecting the vertexes O and R and theconnecting line connecting the vertexes P and Q) is selected; and thesecond camera is mounted in the middle of the A pillar, facing thedirection of the connecting line of these two center points, and thenacquires the second image within the first range on the outside of thecar pillar. Subsequently, the shielding area is determined according tothe position of the human eyes and the A pillar; the image of theshielding area is selected out of the second image; and the selectedimage of the shielding area is displayed in the display mounted on the Apillar.

In another example of the present disclosure, the processor 230 mayfurther include a storage device, which can store, for instance, theabove segmented image of the shielding area in real time. Thus, thevehicle auxiliary display device provided by the embodiment of thepresent disclosure may also have the function of a tachograph.

An embodiment of the present disclosure determines the position of theuser and the monitoring range on the outside of the A pillar accordingto the images acquired by the first camera and the second camera,extracts the image of the visual blind area due to the A pillar from thecaptured image within the monitoring range on the outside of the Apillar according to the position of the user, displays the image of thevisual blind area due to the A pillar on the display on the A pillar,allows the user to “view” the condition in the area shielded by the Apillar through the display, and hence solves the blind area problemcaused by the A pillar and improves the degree of driving safety.

FIGS. 5a-5g are schematic diagrams illustrating the monitoring modes ofthe second camera in another embodiment of the present disclosure. Asshown in FIG. 5a , as the position of the A pillar 110 is mostly notcompletely perpendicular to the plane provided with the car and has aspecific oblique angle f with respect to the direction perpendicular tothe plane provided with the car, and the current display technologymainly aims to obtain rectangular displays due to the utilization rateof glass substrates; a rectangular display may be mounted on the Apillar basically perpendicular to the level ground, but only asmall-sized display can be selected in this case; or the rectangulardisplay may also be mounted on the A pillar along the inclined directionof the A pillar, and a larger display may be selected in this case, butplayed monitoring video is also oblique for users, so the display isunfavorable for the usage of the user. Thus, in order to obtain betterperspective effect, in another embodiment of the present disclosure, thesecond camera 120 is used in monitoring after being inclined by a firstangle relative to the direction perpendicular to the plane provided withthe car, and the display 240 on the A pillar 110 is used for displayafter being inclined by a second angle relative to the directionperpendicular to the plane provided with the car. In this case, althoughthe image of the shielding area of the A pillar displayed by the display240 is oblique relative to the display 240, the observed image is notoblique relative to the direction perpendicular to the plane providedwith the car. That is to say, as for the user, the image observed by theuser is not oblique. It should be noted that the plane provided with thecar is not a plane perpendicular to the gravity and changes along withthe position of the car; and correspondingly the plane provided with thecar is a surface of a slope when the car is driving on the slope.

FIG. 5b illustrates a second image obtained by conventional shooting,namely an image obtained by shooting along the horizontal and verticaldirections. FIG. 5c illustrates the second image obtained byconventional shooting and displayed by the display 240 which isregularly placed (namely horizontally and vertically placed). FIG. 5dillustrates the display effect of the second image obtained byconventional shooting but displayed by the display obliquely arranged onthe A pillar. It can be seen that the image viewed by the user in thiscase is oblique, which is not conducive to the recognition of the imagecontent by the user. In the example of the embodiment of the presentdisclosure, as shown in FIG. 5e , firstly, the second camera 120 isadopted for shooting after rotating by a first angle (e.g., rotating bythe f angle) relative to the direction perpendicular to the planeprovided with the car. Compared with the image in FIG. 5b obtained byconventional shooting, the captured second image is also inclined by thefirst angle relative to the side of the rectangular image. As shown inFIG. 5f , if the content captured in FIG. 5e is displayed on theregularly placed display, the displayed content is inclined by the firstangle relative to four sides of the display. However, the display 240obliquely arranged on the A pillar rotates by a second angle. Thus, asshown in FIG. 5g , if the content captured in FIG. 5e is displayed onthe display, although the displayed content is still oblique relative tothe four sides of the display, the content is horizontally andvertically arranged relative to the user and is consistent or basicallyconsistent with the content viewed in a normal display mode. Herein, thesecond angle is preferably the same as the first angle. Even if thefirst angle and the second angle are different, the difference betweenboth is also preferably less than, e.g., 5°. Thus, although the imageviewed by the user on the display 240 on the A pillar is obliquerelative to the display to some degree, the image is not obliquerelative to the ground provided with the car, namely being basically thesame as the normal viewing angle.

The embodiment of the present disclosure can effectively adjust thedisplay angle of the display image via physical rotation of the secondcamera and the display, provides convenience for the user to rapidlyrecognize the image of the shielding area, further improves the safetydegree, can also reduce amount of system data processing, adopts thedisplay to replace an expensive anomalous screen, and saves theproduction cost.

In addition, an embodiment of the present disclosure further provides avehicle, for instance, a car, which comprises the vehicle auxiliarydisplay device provided by any of the above embodiments. In the car, atleast one pillar is equipped with the vehicle auxiliary display device.For instance, one A pillar is equipped with the vehicle auxiliarydisplay device, or two A pillars are each equipped with the vehicleauxiliary display device, or a B pillar and a C pillar are also equippedwith the vehicle auxiliary display device. The vehicle auxiliary displaydevice is the same as any of the above embodiments. No furtherdescription will be given here. Moreover, for instance, when both two Apillars are equipped with the vehicle auxiliary display device, for boththe two A pillars the first camera may be the same but the secondcameras are different.

In order to mount the vehicle auxiliary display device in the carpillar, an embodiment of the present disclosure also improves thestructure of the car pillar. Description will be given below only bytaking the A pillar of the car as an example. FIG. 6 is a schematicstructural view illustrating the inside of the A pillar of the car inthe embodiment of the present disclosure. As shown in FIG. 5, the Apillar 110 includes a body 610, a barrier part 611, a safety airbag 612and a display 240. The barrier part 611 isolates the safety airbag 612from the display 240, so as to effectively prevent the display frombeing burst and hurting the staff in the car when the safety airbag popsup.

In addition, the A pillar may further include a shield 620. The shield620 is shown by dotted lines in FIG. 6 and is in an open state. Theshield 620 covers the body 610 of the A pillar in a normal mountingstate and can be turned on or off relative to the body 610 of the Apillar. For instance, a fixed rotating shaft 613 is also disposed on oneside of the body 610, and the shield 620 is connected to the fixedrotating shaft 613 and rotates relative to the fixed rotating shaft 613and then turned on or off relative to the body 610.

In addition, the other side of the shield 620 can be fastened with thebody 610 through a fastening connector. When the shield 620 is turned onrelative to the body 610, the fastening connector is switched on. Inthis case, the shield 620 may be conveniently turned on when required tobe turned on, and may also be integrally formed with the body 610 whennot required to be turned on.

In addition, an opening 621 is disposed at a position of the shield 620corresponding to the display 240. A display area of the display 240 maybe exposed at the opening position, so as to provide convenience for theviewing of the user. Moreover, other components of the vehicle auxiliarydisplay device, e.g., a processor and a circuit, may also be hidden inthe A pillar, for instance, disposed on the inside of the barrier part611. The processor of the vehicle auxiliary display device may beimplemented by an independent processor or may be implemented by, forinstance, a central control computer of the car. The implementation isnot limited in the embodiment of the present disclosure.

An embodiment of the present disclosure can mount the vehicle auxiliarydisplay device in the A pillar by the design of the inside of the Apillar, without affecting the strength of the A pillar and occupyingother space in the car, being both artistic and practical.

An embodiment of the present disclosure further provides a displaymethod of a vehicle auxiliary display device. The display method may bea working method of the vehicle auxiliary display device provided by theabove embodiment and is only simply described herein. The specificcontent may refer to the description in the present disclosure.

FIG. 7 is a flow diagram of the display method of the vehicle auxiliarydisplay device, provided by the embodiment of the present disclosure. Asillustrated in FIG. 7, the display method 700 of the vehicle auxiliarydisplay device may comprise the following steps S701-S705:

Step S701: adopting a first camera to monitor and acquire a first imageof the position of a user, for instance, adopting the first camera tomonitor and acquire the first image according to a first spacedetermined according to an area provided with the main driving seat andthe vertical height of the A pillar.

Step S702: adopting a second camera to acquire a second image within afirst range on the outside of a car pillar, for instance, adopting thesecond camera to acquire the second image according to a second spaceformed by extension lines from vertexes of the first space to vertexesof the A pillar.

Step S703: determining a shielding area of the car pillar for shieldingthe user's line of sight based on the first image. For instance, whenthe first camera is adopted to monitor and acquire the first imageincluding an image of the user's eyes in the step S701, in the stepS703, the position of the user's eyes is determined based on the firstimage; the coordinate of the position of the user's eyes in the firstspace is determined according to the position of the user's eyes; andthe shielding area of the A pillar shielding the user's line of sight isdetermined according to the coordinate.

Step S704: extracting an image of the shielding area from the secondimage based on the second image, for instance, determining extensionlines from the coordinate to the vertexes of the A pillar, anddetermining the shielding area of the A pillar shielding the user's lineof sight based on the extension lines, or for instance, determiningmapped coordinates of the extension lines mapped into the second image,and extracting the image of the shielding area from the second imagebased on the mapped coordinates.

Step S705: sending the extracted image of the shielding area to thedisplay for display.

Of course, as described above, the display may also be not used, but theimage of the shielding area is sent to a projector for projection orsent to VR or AR glasses for display.

For the image displayed on the display to be compatible with the viewingangle of the user, according to one example of the present disclosure,the second camera acquires the second image after rotating by a firstangle relative to the direction perpendicular to the plane provided withthe car, and the display displays the image of the shielding area of theA pillar after rotating by a second angle relative to the directionperpendicular to the plane provided with the car, so that the image ofthe shielding area of the A pillar displayed on the display can beinclined relative to the edge of the display and not inclined relativeto the image in the second image.

An embodiment of the present disclosure reflects the external conditionat the blind area of corresponding A pillar onto the display of the Apillar in real time after image processing according to the viewingposition of the human eyes through the monitoring of the first andsecond cameras, achieves the perspective effect of the A pillar, andimproves the driving safety.

It should be understood by those skilled in the art that the units andthe algorithm in the examples of the embodiments of the presentdisclosure can be implemented by electronic hardware, firmware, softwareor a combination thereof. Moreover, a software module may be placed intoa computer storage medium in any form. In order to clearly illustratethe interchangeability of hardware and software, the components and thesteps in the examples have been generally described in the abovedescription according to functions. Whether these functions are executedby hardware or software depends on specific applications and designconstraints of the technical proposal. Each specific application mayimplement the described function by different methods by those skilledin the art, but the implementation shall not exceed the scope of thepresent disclosure.

It should be understood by those skilled in the art that variousmodifications, combinations, partial combinations and replacements maybe made to the present disclosure based on the design demand and otherfactors, as long as they fall within the scope of the appended claimsand equivalent characteristics thereof.

The application claims priority to the Chinese patent application No.201610721420.8, filed on Aug. 24, 2016, the entire disclosure of whichis incorporated herein by reference as part of the present application.

What is claimed is:
 1. A vehicle auxiliary display device, comprising: afirst camera configured to acquire a first image of a position of auser; a second camera configured to acquire a second image within afirst range on an outside of a pillar of a vehicle; a processorconfigured to determine a shielding area of the pillar shielding theuser's line of sight based on the first image, extract an image of theshielding area from the second image based on the second image, and sendthe image of the shielding area to the user; and a display configured todisplay the image of the shielding area, wherein the second camera formsa first angle with respect to the direction perpendicular to the planeprovided with the vehicle and is configured to acquire the second image,the display forms a second angle with respect to the directionperpendicular to the plane provided with the vehicle, the display isdisposed on an A pillar of the pillar, so that the image of theshielding area displayed on the display can be inclined relative to anedge of the display but not inclined relative to the directionperpendicular to the plane provided with the vehicle, and the firstangle, the second angle and an angle of the A pillar relative to thedirection perpendicular to the plane provided with the vehicle are allequal, the first camera is configured to acquire the first imageaccording to a first space determined by an area provided with a maindriving seat and a height of an A pillar as the pillar in a directionperpendicular to a plane provided with the vehicle, and the secondcamera is configured to acquire the second image according to a secondspace defined by first extension lines of connecting lines of at leastpart of vertexes of the first space and at least part of vertexes of theA pillar.
 2. The device according to claim 1, wherein the first imageacquired by the first camera includes an image of the user's eyes; andthe processor determines a position of the user's eyes based on thefirst image, determines a coordinate of the user's eyes in the firstspace based on the position of the user's eye, and determines theshielding area of the A pillar shielding the user's line of sight basedon the coordinate.
 3. The device according to claim 2, wherein theprocessor determines second extension lines of connecting lines from thecoordinate to the vertexes of the A pillar, and determines the shieldingarea of the A pillar shielding the user's line of sight based on thesecond extension lines.
 4. The device according to claim 3, wherein theprocessor determines vertexes of the shielding area in the second imageaccording to the second extension lines, and extracts the image of theshielding area from the second image based on the vertexes of theshielding area.
 5. A vehicle, comprising: pillars and the vehicleauxiliary display device according to claim 1, wherein one of thepillars is equipped with a display of the vehicle auxiliary displaydevice.
 6. The vehicle according to claim 5, wherein the pillars includeA pillars; and the vehicle auxiliary display device is disposed on atleast one A pillar on the front of the vehicle.
 7. The vehicle accordingto claim 6, wherein the at least one A pillar includes a body; the bodyincludes a barrier part and a safety airbag; and the barrier part isconfigured to isolate the safety airbag from the display.
 8. The vehicleaccording to claim 7, wherein the at least one A pillar further includesa shield; and the shield is capable of covering the body of the A pillarand is able to be turned on or off relative to the body of the A pillar.9. The vehicle according to claim 8, wherein an opening is disposed at aposition of the shield corresponding to the display and configured toexpose a display area of the display.
 10. The device according to claim1, wherein the display is a rectangular display.
 11. A display method ofa vehicle auxiliary display device, the vehicle auxiliary display devicecomprising a first camera and a second camera, the display methodcomprising: adopting the first camera to acquire a first image of aposition of a user; adopting the second camera to acquire a second imagewithin a first range on an outside of a pillar; determining a shieldingarea of the pillar shielding the user's line of sight based on the firstimage, and extracting an image of the shielding area from the secondimage based on the second image; sending the extracted image of theshielding area to the user; and displaying the image of the shieldingarea on a display, wherein the second camera forms a first angle withrespect to the direction perpendicular to the plane provided with thevehicle and is configured to acquire the second image, the display formsa second angle with respect to the direction perpendicular to the planeprovided with the vehicle, the display is disposed on an A pillar of thepillar, so that the image of the shielding area displayed on the displaycan be inclined relative to an edge of the display but not inclinedrelative to the direction perpendicular to the plane provided with thevehicle, and the first angle, the second angle and an angle of the Apillar relative to the direction perpendicular to the plane providedwith the vehicle are all equal, the first camera is adopted to acquirethe first image according to a first space determined by an areaprovided with a main driving seat and a height of an A pillar as thepillar in a direction perpendicular to the plane provided with thevehicle, and the second camera is adopted to acquire the second imageaccording to a second space formed by first extension lines ofconnecting lines of at least part of vertexes of the first space and atleast part of vertexes of the A pillar.
 12. The display method accordingto claim 11, wherein the first camera is adopted to acquire the firstimage including an image of the user's eyes, determine the position ofthe user's eyes based on the first image, determine a coordinate of theposition of the user's eyes in the first space based on the position ofthe user's eye, and determine the shielding area of the A pillarshielding the user's line of sight based on the coordinate.
 13. Thedevice according to claim 11, wherein the display is a rectangulardisplay.